Depth-assisted auto focus

ABSTRACT

An apparatus configured for image processing comprises one or more processors configured to determine data associated with a distance between an object and the apparatus and determine a plurality of lens positions of a camera lens based on the data associated with the distance between the object and the apparatus. The one or more processors are further configured to determine, for each one of the plurality of lens positions, a respective focus value to generate a plurality of focus values. To determine, for each one of the plurality of lens positions, the respective focus value, the one or more processors are configured to determine, for each one of the plurality of lens positions, phase difference information. The one or more processors are further configured to determine a final lens position based on the plurality of focus values.

PRIORITY CLAIM

This application claims the benefit of U.S. Provisional PatentApplication No. 63/090,977, filed 13 Oct. 2020, the entire contents ofwhich is incorporated herein by reference.

TECHNICAL FIELD

This disclosure relates generally to systems and methods for imagecapture devices, and specifically to depth-assisted autofocus.

BACKGROUND

Digital cameras use a camera lens to focus incoming light onto a camerasensor for capturing digital images. The camera lens has a focal lengththat places a range of depth of the scene in focus. Portions of thescene closer or further than the range of depth may be out of focus, andtherefore appear blurry in a resulting image. The distance of the cameralens from the camera sensor indicates the distance of the range of depthfor the scene from the camera lens that is in focus. Many devices arecapable of moving the camera lens to adjust the distance between thecamera lens and the camera sensor, thereby adjusting which portions of ascene appear in focus for captured images.

A device may attempt to determine the position of the camera lens toplace a portion of interest of the scene in focus. For example, thedevice may automatically determine a portion of the scene to be in focus(such as the center of a camera's field of view). In response, thedevice may perform autofocus (AF) operations to automatically adjust thecamera lens position so that the portion of the scene is in focus forsubsequent image captures.

SUMMARY

In general, this disclosure describes techniques for performing anautomatic focus (e.g., “autofocus” or “AF”) operation to adjust a cameralens position. This disclosure further describes techniques for usingactive-depth assisted techniques (e.g., laser autofocus, time-of-flight,etc.) together with image-based focus techniques for determining thelens position of a camera. In some examples, the techniques of thisdisclosure may reduce a time to determine a focus point of an autofocusoperation, which may potentially reduce a processing burden on thecamera processor and/or reduce a power consumption of the cameraprocessor. Moreover, reducing a time to determine a focus point of anautofocus operation may improve a user experience. Additionally, in someexamples, the techniques of this disclosure may reduce a time todetermine a focus point of an autofocus operation in low lightconditions.

In one example, an apparatus is configured for image processing. Theapparatus comprises one or more processors configured to determine dataassociated with a distance between an object and the apparatus anddetermine a plurality of lens positions of a camera lens based on thedata associated with the distance between the object and the apparatus.The one or more processors are further configured to determine, for eachone of the plurality of lens positions, a respective focus value togenerate a plurality of focus values. To determine, for each one of theplurality of lens positions, the respective focus value, the one or moreprocessors are configured to determine, for each one of the plurality oflens positions, phase difference information. The one or more processorsare further configured to determine a final lens position based on theplurality of focus values.

In another example, a method for image processing includes determining,with one or more processors implemented in circuitry, data associatedwith a distance between an object and a camera lens and determining,with the one or more processors, a plurality of lens positions of thecamera lens based on the data associated with the distance between theobject and the camera lens. The method further includes determining,with the one or more processors and for each one of the plurality oflens positions, a respective focus value to generate a plurality offocus values. Determining, for each one of the plurality of lenspositions, the respective focus value comprises determining, for eachone of the plurality of lens positions, phase difference information.The method further includes determining, with the one or moreprocessors, a final lens position based on the plurality of focusvalues.

In one example, a device for image processing, the device comprisesmeans for determining data associated with a distance between an objectand the device and means for determining a plurality of lens positionsof a camera lens based on the data associated with the distance betweenthe object and the device. The device further comprises means fordetermining, for each one of the plurality of lens positions, arespective focus value to generate a plurality of focus values. Themeans for determining, for each one of the plurality of lens positions,the respective focus value comprises means for determining, for each oneof the plurality of lens positions, phase difference information. Thedevice further comprises means for determining a final lens positionbased on the plurality of focus values.

In another example, a computer-readable storage medium having storedthereon instructions that, when executed, configure one or moreprocessors to determine data associated with a distance between anobject and a camera lens and determine a plurality of lens positions ofthe camera lens based on the data associated with the distance betweenthe object and the camera lens. The instructions further cause the oneor more processors to determine, for each one of the plurality of lenspositions, a respective focus value to generate a plurality of focusvalues. The instructions that cause the one or more processors todetermine, for each one of the plurality of lens positions, therespective focus value further cause the processor to determine, foreach one of the plurality of lens positions, phase differenceinformation. The instructions further cause the one or more processorsto determine a final lens position based on the plurality of focusvalues.

The details of one or more examples are set forth in the accompanyingdrawings and the description below. Other features, objects, andadvantages will be apparent from the description, drawings, and claims.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram of an example device for performing AF, inaccordance with the techniques of the disclosure.

FIG. 2A is a depiction of an example camera lens at a focal length sothat an object is in focus for autofocus (AF).

FIG. 2B is a conceptual diagram of an example camera lens at too long ofa focal length so that the object is out of focus for contrast detectionAF.

FIG. 2C is a conceptual diagram of an example camera lens at too shortof a focal length so that the object is out of focus for contrastdetection AF.

FIG. 3 is a conceptual diagram of an example correlation between focallength and contrast for contrast detection AF.

FIG. 4 is a conceptual diagram of an example of a contrast detectionautofocus technique.

FIG. 5A is a conceptual diagram of a contrast detection autofocustechnique.

FIG. 5B is a conceptual diagram of an active-depth assisted focustechnique.

FIG. 5C is a conceptual diagram of a phase detection (PD) autofocustechnique.

FIG. 6A is a conceptual diagram of active-depth assisted focus features.

FIG. 6B is a graph diagram of active-depth assisted focus features.

FIG. 7 is a block diagram of a concurrent active-depth assisted focusalgorithm, in accordance with the techniques of the disclosure.

FIG. 8A is a flowchart showing an example technique for an active-depthassisted focus with concurrent image-based refinement, in accordancewith the techniques of the disclosure.

FIG. 8B is a conceptual diagram showing an example technique for anactive-depth assisted focus with concurrent image-based refinement, inaccordance with the techniques of the disclosure.

FIG. 9A is a conceptual diagram showing an example of least squarepolynomial fitting, in accordance with the techniques of the disclosure.

FIG. 9B is a conceptual diagram showing example details of a fine searchcontroller configured for an active-depth assisted focus with concurrentimage-based refinement, in accordance with the techniques of thedisclosure.

FIG. 10 is a conceptual diagram showing two example cases for an examplefine search stage for an active-depth assisted focus with concurrentimage-based refinement, in accordance with the techniques of thedisclosure.

FIG. 11 is a conceptual diagram showing an example process for anactive-depth assisted focus with concurrent image-based refinement thatdecides whether to skip a fine search, in accordance with the techniquesof the disclosure.

FIG. 12 is a conceptual diagram of an example phase detection auto focus(PDAF) sensor.

FIG. 13 is a conceptual diagram showing one example technique fordetermining phase difference pixel data in a PDAF sensor.

FIG. 14 is a flowchart illustrating an example method of operationaccording to one or more example techniques described in thisdisclosure.

DETAILED DESCRIPTION

During an autofocus (AF) operation for a camera, a position of acamera's lens with reference to the camera's image sensor may beadjusted to place a portion of the scene in focus for the camera. Forexample, an AF operation may be performed to move the camera lens inorder to place one or more objects in a region of interest (ROI) of ascene in focus (e.g., the portion of the scene received by a center ofthe image sensor is in focus in resulting images). An AF operation maybe triggered if a camera is being initialized or if a scene has changed.Scene change examples include the camera moving (which may be referredto herein as global motion) or objects in the scene moving (which may bereferred to herein as local motion). Examples of global motion include auser rotating the camera, shaking the camera, or the camera otherwisemoving during operation. Examples of local motion include a personwalking in front of the camera, people, cars, or other objects in thescene moving during frame capture, a user's finger covering a portion ofthe camera lens or camera aperture to the image sensor, or other changesto the scene during camera operation.

An example AF operation may comprise contrast detection (CD) autofocus(CDAF or contrast detection AF). For CDAF, a frame may be received bythe camera's image sensor, and a contrast may be determined for aportion of the scene from the frame. The camera lens position isadjusted until a suitable contrast associated with the camera lensposition is identified. The contrast is a difference in measured lightintensity (e.g., luminance) between neighboring pixels of the imagesensor. For a region of an image that is not in focus (which may bereferred to as out-of-focus), contrast values overall may be negligibleor below a threshold as the light intensity may be similar forneighboring pixels. For example, there exists less variation in lightintensity between pixels when a portion of an image is blurry (e.g.,out-of-focus) than when the portion of the image is clear (e.g.,in-focus). A suitable contrast (which may be associated with a specificlens position) may be greater than a threshold contrast or may begreater than contrasts associated with neighboring lens positions. Forexample, the camera lens may be moved to a focus position associatedwith a largest contrast for a region of interest (ROI) in the scene(which may be referred to as a peak contrast).

Another example AF operation comprises an active-depth assisted focusoperation (e.g., laser autofocus, time-of-flight, etc.). In someexamples, an active-depth assisted operation may include thetransmittance of near-infrared/infrared electromagnetic (NIR/IR EM)spectrum. An active-depth assisted operation may include atime-of-flight (ToF) calculation.

For example, one or more sensors (e.g., laser sensors) can return dataassociated with the distance between an object and a device and/orcamera lens, such as for instance, a physical distance for every frame.In some examples, the distance is measured on the magnitude ofmillimeters. In another example, the distance is determined based on aToF calculation, e.g., the time difference between emitting light anddetecting returned or reflected light.

At the start of a monitor phase, a device (e.g., an imaging device) maydetermine data associated with the distance between an object and thedevice and/or camera lens of the device. For example, the device maydetermine the distance between the device and a scene of a referentialframe is determined, and when the difference between the distancesdetected between a subsequent frame and the referential frame is greaterthan some threshold, a scene change is detected. For example, similar tothe luma value and movement based determinations, a threshold can be setto 100 mm. At the first frame of the monitor phase the distance of theobject may be determined to be 200 mm from the imaging device, and thenat some subsequent frame the distance from the imaging device to theobject is determined to be 600 mm. Thus, the difference in the distancesof the referential and subsequent frame is greater than the threshold,and the imaging device is configured to determine that the scene haschanged. While the preceding example included a threshold set to 100 mm,any threshold value may be selected based on the desired sensitivity ofthe scene change determination.

An example of another AF operation includes phase detection auto focus(PDAF). In some examples, an image sensor configured for PDAF mayinclude dual photodiode pixels. Such image sensors may be referred to asdual PDAF sensors or dual photodiode (2PD) sensors. In some example dualPDAF sensors, two streams of data may be sent to a camera processor. Onestream includes the pixel data to be processed when capturing a stillimage or video. The other stream includes phase difference pixel dataused in a phase detection auto focus process.

In accordance with the techniques of the disclosure, an autofocustechnique is described herein that uses an active-depth assisted focus(e.g., laser AF, ToF, etc.) along with an image-based focus refinement.Moreover, techniques described herein may perform image-based focusrefinement using contrast difference information and/or a phasedifference information from images acquired at the same time as the ToFmeasurements. The contrast information and/or phase differenceinformation may be used to determine whether or not further fine searchtechniques are needed, which may reduce or eliminate a fine searchprocess relative to other laser autofocus techniques. In this way, adevice may be configured to perform autofocus at a faster speed andaccuracy compared to systems that rely on an active-depth assisted focus(e.g., laser AF, ToF, etc.) and further fine search refinement, CDAF, orPDAF.

FIG. 1 is a block diagram of an example device 100 configured to performan adjustable AF operation. Device 100 may include camera 102, one ormore camera processors 110 (also referred to herein as simply “one ormore processors 110”), and a memory 106 storing instructions 108. One ormore processors 110 may include, for example, an image processor 112 anda processor 104. In some examples, device 100 may include (or be coupledto) a display 114 and/or a number of input/output (I/O) components 116.Device 100 may also include a power supply 118, which may be coupled toor integrated into device 100. Device 100 may include additionalfeatures or components not shown. For example, a wireless interface,which may include a number of transceivers and a baseband processor, maybe included for a wireless communication device. In another example,device 100 may include or be coupled to additional cameras other thancamera 102.

Camera 102 may be capable of capturing individual image frames (e.g.,still images) and/or capturing video (e.g., a succession of receivedimage frames). Camera 102 may include one or more camera lenses 122(also referred to herein as simply “camera lens 122”) and one or moreimage sensors 124 (also referred to herein as simply “image sensors124”). Image sensor 124 may include processing circuitry, an array ofpixel sensors (e.g., pixels) for capturing representations of light,memory, such as buffer memory or on-chip sensor memory, etc. Each one ofone or more image sensors 124 may be coupled with a different one of oneor more camera lenses 122, each lens and image sensor combination havingdifferent apertures and/or fields-of-view. Example lenses may include atelephoto lens, a wide angle lens, an ultra-wide angle lens, or otherlens types.

One or more step controllers 123 (also referred to herein as simply“step controller 123”) may move camera lens 122 to adjust a focal lengthfor image sensor 124. During AF, one or more processors 110 may beconfigured to control movement of camera lens 122 to a final lensposition and/or focus position (e.g., by causing an actuator to movecamera lens 122). For example, device 100 may control a voice-coil motor(VCM) to move camera lens 122 to a determined focus position.

Transmitter 101 may be configured to transmit a signal for active depthsensing. Transmitter 101 may transmit pulsed light or a modulated lightsignal at a defined frequency for time-of-flight (ToF) depth sensing. Inthis manner, transmitter 101 may be referred to as being part of atime-of-flight (ToF) system. Image sensor 124 may be configured toreceive reflections of the light for ToF depth sensing. For example,transmitter 101 may transmit a signal and image sensor 124 may receive areflection of the signal. In this example, one or more processors 110may determine a round trip time or a phase difference (PD) between thetransmitted signal and the received signal. One or more processors 110may associate the round trip time or phase difference with a depth of anobject based on the speed of the signal (e.g., the speed of light for alight signal).

The transmitted signal may be light. The transmitted light may be in anysuitable spectrum, such as the visible light spectrum or outside thevisible light spectrum. For example, the transmitted light may beinfrared (IR) light. As used herein, IR light may refer to light in thevisible light spectrum (such as close to the red color light spectrum)or IR light outside the visible light spectrum (which may includeshortwave IR (SWIR) light, near-field IR (NIR) light, far-field IR (FIR)light, and so on). The light may include ultraviolet light. The signalmay include a frequency outside the light spectrum, such as a radiosignal, sound signal, and so on. Image sensor 124 may be configured toreceive such signals outside the light spectrum. For example, for radiosignals, image sensor 124 may include a radio receiver. Therefore, whilethe examples describe active depth sensing using transmitted light, thepresent disclosure is not limited to a ToF system using light.Additionally, while ToF depth sensing is described as an example activedepth sensing, any suitable active depth sensing system may be used. Forexample, a structured light depth sensing system, sound detection andranging (SONAR), light detection and ranging (LIDAR), radio detectionand ranging (RADAR), or any other suitable active depth sensing systemmay be used.

One or more processors 110 may be configured to control camera 102 andcontrol an active depth sensing system (such as transmitter 101). One ormore processors 110 may include a processor 104 and an image processor112. Processor 104 may be one or more suitable processors capable ofexecuting scripts or instructions of one or more software programs (suchas instructions 108) stored within memory 106. In some aspects,processor 104 may be one or more general purpose processors that executeinstructions 108 to cause device 100 to perform any number of functionsor operations. In additional or alternative aspects, processor 104 mayinclude integrated circuits or other hardware to perform functions oroperations without the use of software.

Image processor 112 may be configured to process received image framesor video provided by camera 102. Image processor 112 may executeinstructions from a memory (such as instructions 108 from memory 106 orinstructions stored in a separate memory coupled to image processor112). Image processor 112 may include specific hardware. Image processor112 may alternatively or additionally include a combination of specifichardware and the ability to execute software instructions.

Image processor 112 may determine information associated with a depth ofan object based on active depth sensing (e.g., using an RTT and/or aphase difference between transmitted light and received light). As notedabove, while ToF depth sensing is described, other implementations ofdepth sensing may be used. For example, image processor 112 may performdepth sensing using structured light (which may include a knowndistribution of light transmitted, received, and compared to determineone or more depths of objects). In another example, transmitter 101 mayprovide a flood illumination, and intensities of the receivedreflections of the flood illumination may be used to determine one ormore depths of objects. Transmitter 101 may also be referred to as a“projector” or an “emitter,” and is not limited to a specifictransmission component. Image sensor 124 may be referred to as a“detector,” “sensor,” “sensing element,” or “photodetector,” and is notlimited to a specific receiving component. While examples describedherein may describe image processor 112 as processing information fromcamera 102, in some examples, image processor 112 may be omitted andprocessor 104 may instead perform the steps performed by image processor112.

Memory 106 may be a non-transient or non-transitory computer readablemedium storing computer-executable instructions 108 to perform all or aportion of one or more operations described in this disclosure. In someexamples, memory 106 may be configured to store one or more sets ofanchor points.

While shown to be coupled to each other via the processor 104 in theexample of FIG. 1, processor 104, memory 106, one or more processors110, optional display 114, and optional I/O components 116 may becoupled to one another in various arrangements. For example, processor104, memory 106, one or more processors 110, optional display 114,and/or optional I/O components 116 may be coupled to each other via oneor more local buses (not shown for simplicity).

Display 114 may be any suitable display or screen allowing for userinteraction and/or to present items (such as received images, video, ora preview image) for viewing by a user. In some aspects, display 114 maybe a touch-sensitive display. I/O components 116 may be or include anysuitable mechanism, interface, or device to receive input (such ascommands) from the user and to provide output to the user. For example,I/O components 116 may include (but are not limited to) a graphical userinterface, keyboard, mouse, microphone and speakers, and so on. Display114 and/or the I/O components 116 may provide a preview image to a userand/or receive a user input for adjusting one or more settings of camera102. For example, display 114 may be used by a user for selecting and/ordeselecting an ROI of the scene (associated with a portion of imagesensor 124) by touching a portion of a displayed preview image. One ormore processors 110 may use the selected ROI for AF to determine a finalcamera lens position of camera lens 122.

In accordance with the techniques of the disclosure, image processor 112may be configured to determine data associated with a distance betweenobject 126 and device 100. Data associated with a distance betweenobject 126 and device 100 may include one or more depth values stored indisparity terms and/or at quantized levels. Device 100 may determine aplurality of lens positions of camera lens 122 for camera 102 based onthe data associated with the distance between object 126 and device 100.In some examples, to determine the plurality of lens positions of cameralens 122, image processor 112 may perform a coarse search. For example,image processor 112 may apply, with transmitter 101 and receiver 103, aToF depth sensing process and/or laser AF process to determine thedistance. Image processor 112 may be configured to determine, for eachone of the plurality of lens positions, a respective focus value togenerate a plurality of focus values. To determine, for each one of theplurality of lens positions, the respective focus value, image processor112 is configured to determine, for each one of the plurality of lenspositions, phase difference information (see FIGS. 12, 13). Determininga focus value for each one of a plurality of lens positions using aphase difference may improve an accuracy of the focus values compared tosystems relying only on CDAF. Improving an accuracy of the focus valuesmay increase a speed of AF, which may improve a user experience relatingto a photography.

Image processor 112 may be further configured to determine a final lensposition based on the plurality of focus values. For example, imageprocessor 112 may determine to skip a fine search based on the focusvalue of a particular lens position satisfying a focus threshold anddetermine the final lens position as the particular lens position.Skipping the fine search based on the focus value of a particular lensposition satisfying a focus threshold may help to reduce a time todetermine a focus point of an autofocus operation compared to systemsthat always perform a fine search, which may potentially reduce aprocessing burden on image processor 112 and/or reduce a powerconsumption of image processor 112. Moreover, reducing a time todetermine a focus point of an autofocus operation may improve a userexperience. Additionally, in some examples, the techniques of thisdisclosure may reduce a time to determine a focus point of an autofocusoperation in low light conditions.

In some examples, image processor 112 may determine a configuration fora fine search (e.g., a starting lens position, a direction of search,and/or a search range) based on the focus value of each respective lensposition of the plurality of lens positions and may apply the finesearch using the configuration. For instance, image processor 112 mayapply a CDAF search using the configuration. Performing a fine searchwith a configuration determined using focus values for lens positionsused to determine a distance may help to reduce a time to determine afocus point of an autofocus operation compared to systems that perform afine search without configuration information, which may potentiallyreduce a processing burden on image processor 112 and/or reduce a powerconsumption of image processor 112. Moreover, reducing a time todetermine a focus point of an autofocus operation may improve a userexperience. Additionally, in some examples, the techniques of thisdisclosure may reduce a time to determine a focus point of an autofocusoperation in low light conditions.

FIG. 2A is a depiction of a camera 102 with a camera lens 122 at a focallength 128A from the image sensor 124 so that an object 126 is in focusat focus distance 120. FIG. 2B is a depiction of the camera 102 with thecamera lens 122 at too long of a focal length 128B so that the object126 is out of focus. FIG. 2C is a depiction of the camera 102 with thecamera lens 122 at too short of a focal length 128C so that the object126 is out of focus.

The illustrated bumps/curves indicate the distribution of measuredluminance of light from the object 126 and refracted by the camera lens122 toward the image sensor 124. An object 126 is considered in focuswhen the distribution has a smaller standard deviation (which may bereferred to as being tighter) than the distributions for the other focallengths. As shown, the object 126 is out of focus when the distributionof measured light intensity for the object is spread out compared to thedistribution for the focal length when the object is in focus. Asillustrated, the curve depends on the position of camera lens 122directing the light to the image sensor 124. If camera lens 122 is toofar from the image sensor 124 (such as in FIG. 2B), the light from theobject 126 is directed onto a larger portion of the image sensor 124than if camera lens 122 is closer to the image sensor 124 (such as inFIG. 2A). Similarly, if camera lens 122 is too close to the image sensor124 (such as in FIG. 2C), the light from the object 126 is directed ontoa larger portion of the image sensor 124 than if camera lens 122 iscloser to the image sensor 124 (such as in FIG. 2A). In this manner, thepeak of the curve in FIG. 2A is greater than the peaks of the curves inFIG. 2B and FIG. 2C. While contrast is described as a difference inlight intensity between neighboring pixels, contrast may be a differencein chrominance or a difference in some combination of chrominance andlight intensity.

FIG. 3 is a depiction of an example correlation 300 between focal lengthand contrast for CDAF. Again, CDAF may be an example AF operation. Asshown, the correlation 300 between lens position (e.g., a focal length)and contrast may be parabolic and/or second order. The correlation 300is depicted as a continuous curve of contrast across lens positions. Insome implementations, a correlation may be represented as a stepwisefunction, a dot plot, or other suitable discrete or continuousdistribution. For example, a camera lens may be configured to bepositioned in X number of lens positions. The correlation may include Xnumber of contrasts, with each contrast associated with a lens position.The example correlations in the present disclosure are for the purposeof describing aspects of the disclosure, and the disclosure is notlimited to a specific type of correlation. In addition, the exactcurvature may differ, and the depictions are for illustrative purposes.For example, the correlation 300 may be expressed in general by a secondorder equation y=ax²+bx+c, where the contrast is y, the focal length isx, the curvature of the parabola is indicated by a, the slope of theparabola is indicated by b, and the offset of the parabola is indicatedby c.

The correlation 300 may be associated with an ROI of image sensor 124.For example, the ROI of image sensor 124 may be a center pixel orportion, and an object in a scene received by the center of the imagesensor is associated with the focal lengths. Final lens position 304 forwhich the object is in focus corresponds to the contrast being at amaximum compared to the contrasts for other focal lengths. For theexample correlation 300, the final lens position 304 is associated withthe vertex. For a correlation represented by a second order equationy=ax²+bx+c, the vertex may be −b/2a.

The correlation 300 indicates a relationship between focal lengths andcontrasts measured by the image sensor. However, correlation 300 may notbe known to image processor 112. Instead, image processor 1120, whenperforming CDAF, may cause camera 102 (e.g., with step controller 123)to move camera lens 122 in a recursive manner until image processor 112identifies the peak contrast (which may be associated with the finallens position 304). To perform CDAF, step controller 123 may placecamera lens 122 at an initial position, a frame is received with cameralens 122 positioned, and image processor 112 may determine a contrastfrom the frame for an ROI of image sensor 124. Image processor 112 mayiteratively move, with step controller 123, camera lens 122 (e.g.,forwards and backwards), receive a frame, and determine contrast untildetermining a sufficient contrast (such as a peak contrast). Referringto correlation 300, image processor, with step controller 123, mayposition camera lens 122 at an initial lens position 302 (e.g., atminimum lens position) to begin CDAF and may move camera lens 122.Example movements may include coarse adjustments 306 and fineadjustments 308, with coarse adjustments 308 associated with larger lensposition movements than fine adjustments 306. Referring to thecorrelation 300, coarse adjustments 306 allow an approach to the finallens position 304 in a fewer number of frames than exclusively usingfine adjustments 308, while using fine adjustments 308 allows for lessovershoot and error in determining the final lens position 304. Coarseadjustments 306 and fine adjustments 308 may be fixed or variable insize, pre-configured or configurable, or any other suitable types offocal length adjustments for CDAF.

In the example of FIG. 3, camera 102 may perform course adjustment 306and fine adjustments 308 iteratively until determining final lensposition 304. For example, image processor 112 may determine a contrastfor initial lens position 302, and may perform coarse adjustments 306 toadjust the focal length to intermediate focal lengths 310. Imageprocessor 112 may determine a contrast for each of the intermediatefocal lengths 310 and iteratively perform coarse adjustments 306. Theprocess may repeat until image processor 112 determines one ofintermediate focal lengths 310 is within a threshold of the final lensposition 304. Image processor 112 may repeat the course adjustmentprocess until the contrast is above a threshold. Once above thethreshold, image processor 112 may perform fine adjustments 308. Forexample, image processor 112 may perform coarse adjustments 306 if thedifference between a previous contrast and a current contrast isincreasing, the same as previous differences, or greater than athreshold difference between intermediate focal lengths 310 to convergeto final lens position 304. In some examples, image processor 112 maystop performing coarse adjustments 306 if the difference between aprevious contrast and a current contrast is decreasing, or is less thana threshold difference between intermediate focal lengths 310. Imageprocessor 112 may perform fine adjustments 308 until converging to finallens position 304. As shown, image processor 112 may overshoot finallens position 304 when fine adjustments 308 are converging to final lensposition 304, may overshoot the final lens position 304. As noted, imageprocessor 112 may perform a CDAF technique that is an iterative processof measuring the contrast for a ROI with camera lens 122 at differentfocal lengths, adjusting the focal length, and again measuring thecontrast until determining a final lens position 304. Image processor112 may apply both coarse adjustments 306 and fine adjustments 308,which may reduce the number of frames used for an AF operation comparedto systems exclusively performing fine adjustments. Image processor 112may perform CDAF using 12 or more frames.

FIG. 4 is a conceptual diagram of an example of a contrast auto focustechnique. At each lens stop position, device 100 may determine a focusvalue (FV) is reported. A FV curve 402 can be plotted in terms ofvarious lens stop positions. The higher the focus value is, the betterthe focus will be. The peak of the FV curve 402 may represent the lensstop position where the best (e.g., highest) focus is obtained.

FIG. 5A is a conceptual diagram of a contrast detection autofocustechnique. In the example of FIG. 5A, image processor 112 may perform acourse search operation that uses 12 frames and a fine search operationthat uses 3-5 frames to determine the peak of FV curve 510. In FIG. 5A,the horizontal axis represents a lens position and the vertical axisrepresents a focus measure, such as, a degree of sharpness or degree offocus of an image or image pixel.

Contrast AF=Coarse search (12 frames)+Fine search (3-5 frames)

In general, CDAF may result in overshooting or focus hunting in order toget a reliable focus peak. Compared to active-depth assisted focus andPDAF, CDAF results in a relatively low focus speed, which may lead to apoor user experience that is relatively low.

FIG. 5B is a conceptual diagram of an active-depth assisted focustechnique. In the example of FIG. 5B, image processor 112 may perform acourse search operation that uses 1-2 frames and a fine search operationthat uses 5 frames to determine the peak of the FV curve 510. In FIG.5B, the horizontal axis represents a lens position and the vertical axisrepresents a focus measure, such as, a degree of sharpness or degree offocus of an image or image pixel. Device 100 may use laser AF as anactive-depth assisted focus technique for example purposes only.

Laser AF=Laser Assisted (˜1-2 frames)+Fine search (˜5 frames)

Compared to CDAF and PDAF, active-depth assisted focus comprises a focusspeed that is high and user experience that is relatively good. However,active-depth assisted focus may include focus hunting due to a finesearch. Active-depth assisted focus may provide good results in lowlight environments compared to other AF techniques.

FIG. 5C is a conceptual diagram of a phase detection (PD) autofocustechnique. In the example of FIG. 5C, image processor 112 may perform acourse search operation that uses 1-2 frames and a fine search operationthat uses zero or more frames to determine the peak of the FV curve 510.

PD(Phase Detection) AF=PD AF(˜1-2 frames)+Fine search (optional)

Compared to CDAF and active-depth assisted focus, PDAF comprises a focusspeed that is high and a user experience that is relatively good by nothaving any focus hunting. However, PDAF may not be accurate in low lightenvironments.

FIG. 6A is a conceptual diagram of active-depth assisted focus features.FIG. 6A discusses limitations of active-depth assisted focus.Specifically, FIG. 6A illustrates a distance to lens position and atheory of distance verses a lens position.

As shown in FIG. 6A, a focal length (f) of camera lens 122 defines therelations between a distance (S₁) from camera lens 122 and object 126and a distance (S₂) from camera lens 122 and image sensor 124 of camera102 as described in EQUATION 1.

$\begin{matrix}{{\frac{1}{S_{1}} + \frac{1}{s_{2}}} = \frac{1}{f}} & {{EQUATION}\mspace{14mu} 1}\end{matrix}$

Solving equation 1 for the distance (S₂) from camera lens 122 and imagesensor 124 results in equation 2

$\begin{matrix}{S_{2} = {f*\frac{S_{1}}{S_{1} - f}}} & {{EQUATION}\mspace{14mu} 2}\end{matrix}$

Solving equation 2 for a lens shift in microns (i.e., S₂−f) results inequation 3.

$\begin{matrix}{{S_{2} - f} = {f*\frac{S_{1}}{S_{1} - f}}} & {{EQUATION}\mspace{14mu} 3}\end{matrix}$

Using equation 3, image processor 112 may calculate a lens shift basedon the focal length of camera lens 122 and the distance (S₁) from object126. In this way, image processor 112 may provide AF without relying onCDAF, which may reduce a time to determine a focus point of an autofocusoperation, which may potentially reduce a processing burden on imageprocessor 112 and/or reduce a power consumption of image processor 112.

Several factors make accuracy worse for distance to lens positionmapping of equation 3, such as, for example, a voice-coil motor (VCM)issue (e.g., a gravity offset, hysteresis, etc.), an AF calibration(e.g., an accuracy of inf, macro, etc.), and a temperature (e.g., aneffective focal length (EFL) shift). As such, when using active-depthassisted focus techniques, image processor 112 may apply a fine scanthat takes an additional 5 frames performing lens position mapping. Inthis way, an accuracy of the AF using active-depth assisted focus may beimproved.

FIG. 6B is a graph diagram of active-depth assisted focus features. Inthe example of FIG. 6B, the horizontal axis represents distance (S₁)from camera lens 122 and an object 126 with decreasing values frominfinity and the vertical axes represents distance (S₂) from camera lens122 and image sensor 124. As shown, curve 602 shows that the distance(S₂) from camera lens 122 and image sensor 124 of a camera increases(e.g., moves from 0 to 250) as a distance (S₁) from camera lens 122 andan object 126 decreases (e.g., moves from 600 to 0).

FIG. 7 is a conceptual diagram of a concurrent active-depth assistedfocus algorithm, in accordance with the techniques of the disclosure.Image processor 112 may start a search (e.g., a course search) for afinal lens position (702). For instance, image processor 112 mayinitiate a process to determine the distance (S₂) from camera lens 122and image sensor 124 for an object 126 that is a region of interest. Inthe example of FIG. 7, image processor 112 may perform an active-depthassisted focus, such as, for example, laser autofocus or ToF (704). Forexample, image processor 112 may determine a time-of-flight to determinethe distance (S₁) from camera lens 122 and object 126. In this example,image processor 112 may determine the distance (S₂) from camera lens 122and image sensor 124 for object 126 using equation 3.

Active-depth assisted focus (e.g., step 704) may only output a distanceto object 126. In this example, image processor 112 may determine agenerate a lens position mapping to determine a final lens position.However, the mapping function may be impacted by VCM gravity,hysteresis, and the lens position calibration. As such, image processor112 may apply a fine search (706), such as, for example, CDAF, until theAF process is done (708). For example, image processor 112 may apply thefine search process contrast AF refinement process illustrated in FIG.5B until the FV converges to a peak value. In this way, the contrast AFrefine search may help to mitigate one or more of the VCM issue, the AFcalibration issue, and the temperature issue described in FIGS. 6A, 6B.

One or more problems may exist in active-depth assisted focus followedby a fine scan 710 (e.g., performing coarse search 712 and skippingconcurrent image-based refinement 714). For example, active-depthassisted focus followed by a fine scan 710 may not be as fast as PDAF.For example, because active-depth assisted focus followed by a fine scan710 does not process actual images (e.g., perform concurrent image-basedrefinement 714) during active-depth assisted focus (e.g., step 704.),image processor 112 may add a fine search (e.g., step 706) afteractive-depth assisted focus to ensure that the selected lens position ofcamera lens 122 is accurate, which may lead the active-depth assistedfocus followed by a fine scan process being not as fast as PDAF. Forexample, image processor 112 may cause step controller 123 to movecamera lens 122 backwards and forwards to find the peak FV value.Therefore, the user experience may be poor when applying active-depthassisted focus followed by a fine scan 710 because the autofocus isrelatively slow.

Techniques described herein may address one or more of the aboveproblems and/or one or more other problems. For example, techniquesdescribed herein may configure image processor 112 to use concurrentactive-depth assisted focus techniques, which may improve theactive-depth assisted focus speed and accuracy by adding concurrentimage-based focus refinement.

In accordance with the techniques of the disclosure, image processor 112may apply active-depth assisted focus 704 with a concurrent image-basedfocus refinement (714) until the AF process is done (716). That is,while step controller 123 moves camera lens 122 to perform active-depthassist focus 704, image processor 112 may perform concurrent image-basedrefinement 714 to determine a focus value for a set of images capturedwhen performing active-depth assist focus 704. In this way, fine search706 may be skipped, which may result in the AF being smoother comparedto systems that use active-depth assisted focus 704 followed by finesearch 706. For instance, using active-depth assisted focus 704 withconcurrent image-based focus refinement 714 may allow image processor112 to skip the process of moving camera lens 122 backwards and forwardsto find the peak FV value for fine search 706.

FIG. 8A is a flowchart showing an example technique for active-depthassisted focus with concurrent image-based refinement, in accordancewith the techniques of the disclosure. Image processor 112 may start asearch for a final lens position (802). For instance, image processor112 may initiate a process to determine the distance (S₂) from cameralens 122 and image sensor 124 for object 126 that is a region ofinterest. During a coarse search stage 804, image processor 112 may usean active-depth assisted focus to obtain the converted target distance(e.g., distance from object 126). Image processor 112 may determine thedistance using, for example a laser focus on the target scene. In theexample of FIG. 8A, image processor 112 may apply laser focus, however,image processor 112 may apply any active-depth assisted focus. Forinstance, image processor 112 may apply a ToF algorithm to determine thedistance.

During an image-based focus evaluator stage 806, image processor 112 maycalculate focus values (e.g., contrast and/or phase differenceinformation) from an image at each lens position for camera lens 122.For example, image processor 112 may calculate the focus value based onthe image content for each lens position of camera lens 122. Imageprocessor 112 may perform the focus value calculation using any kind ofa contrast AF algorithm and/or PD algorithm.

During image-based focus evaluator stage 806, image processor 112 mayreceive one or more of: (1) contrast information for a horizontaldirection; (2) contrast information for a vertical direction; and/or (3)PD information. Image processor 112 may calculate one or more of: (1) afitting peak; and (2) a confidence (e.g., the confidence can determinedby the shape of a fitting curve). In some examples, image processor 112may determine the shape of the fitting curve by one or more propertiesof the fitting curve, where the one or more properties of the fittingcurve may include one or more of a shape, a value, and a peak of thefitting curve.

Image processor 112, with step controller 123, may move (e.g., step)camera lens 122 to a next step lens position (808). Image processor 112,with step controller 123, may move to another lens position based on apreconfigured step size. After collecting data from enough steps (e.g.,two or more steps), image processor 112 may use data collected duringthe image-based focus evaluator stage (e.g., contrast information and/orphase difference information) to determine whether or not to skip thefine search (810).

To determine, for each one of the plurality of lens positions, therespective focus value, image processor 112 may be configured todetermine, for each one of the plurality of lens positions, phasedifference information (see FIGS. 12, 13). Determining a focus value foreach one of a plurality of lens positions using a phase difference mayimprove an accuracy of the focus values compared to systems relying onlyon CDAF. Improving an accuracy of the focus values may increase a speedof AF, which may improve a user experience relating to a photography.

For example, image processor 112 may determine to not skip the finesearch (“NO” of step 810) in response to a determination that each focalvalue determined in step 806 is less than a threshold focal value. Insome examples, image processor 112 may determine to skip the fine search(“YES” of step 810) in response to determining that at least one focalvalue determined in step 806 is greater than a threshold focal value.

In response to determining to apply fine search (“YES” of step 810),image processor 112 may apply fine search (812). For example, imageprocessor 112 may apply a contrast AF fine search (see fine search inFIG. 5B). In some examples, image processor 112 may determine aconfiguration for a fine search (e.g., a starting lens position, adirection of search, and/or a search range) based on the focus value ofeach respective lens position of the plurality of lens positions and mayapply the fine search using the configuration. For instance, imageprocessor 112 may apply a CDAF search using the configuration.Performing a fine search with a configuration determined using focusvalues for lens positions used to determine a distance may help toreduce a time to determine a focus point of an autofocus operationcompared to systems that perform a fine search without configurationinformation, which may potentially reduce a processing burden on imageprocessor 112 and/or reduce a power consumption of image processor 112.Moreover, reducing a time to determine a focus point of an autofocusoperation may improve a user experience. Additionally, in some examples,the techniques of this disclosure may reduce a time to determine a focuspoint of an autofocus operation in low light conditions.

In response to determining not to apply the fine search (“NO” of step810), image processor 112 may skip the fine search and use a finalposition from active-depth assist focus (814). For example, imageprocessor 112 may select a lens position of camera lens 122 used duringcoarse search stage 804 that image processor 112 determined to have thelargest focal value (e.g., using contrast and/or phase differencevalues). Skipping the fine search based on the focus value of aparticular lens position satisfying a focus threshold may help to reducea time to determine a focus point of an autofocus operation with littleor no loss to AF accuracy compared to systems that always perform a finesearch, which may potentially reduce a processing burden on imageprocessor 112 and/or reduce a power consumption of image processor 112.Moreover, reducing a time to determine a focus point of an autofocusoperation may improve a user experience. Additionally, in some examples,the techniques of this disclosure may reduce a time to determine a focuspoint of an autofocus operation in low light conditions.

FIG. 8B is a conceptual diagram showing an example technique foractive-depth assisted focus with concurrent image-based refinement, inaccordance with the techniques of the disclosure. Image processor 112,with step controller 123, may perform step 808 of FIG. 8A. In FIG. 8B,the horizontal axis represents a lens position of camera lens 122 andthe vertical axis represents a focus measure, such as, a degree ofsharpness or degree of focus of an image or image pixel captured byimage sensor 124.

Using active-depth assisted focus followed by a fine scan (e.g.,preforming steps 804 and 808 and skipping step 806), image processor112, with step controller 123, may step to lens position 854A based on atime of flight. For instance, image processor 112 may apply a ToFalgorithm to determine the distance and determine lens position 854Abased on the distance. In this example, image processor 112, with stepcontroller 123, may step from lens position 854A to lens position 854B,to lens position 854C, and to lens position 854D to perform a finesearch (e.g., step 812). Image processor 112 may select position lensposition 854A in response to determining that lens position 854A has ahighest focal value on FV curve 855. As noted above, image processor112, with step controller 123, may step camera lens 122 backwards andforwards (e.g., between lens positions 854A-854D) to find the peak FVvalue of FV curve 855. Therefore, the user experience may be poor whenapplying active-depth assisted focus followed by a fine scan because theautofocus is relatively slow.

Techniques described herein may address one or more of the aboveproblems and/or one or more other problems. For example, techniquesdescribed herein may use concurrent active-depth assisted focustechniques, which may improve the active-depth assisted focus speed andaccuracy by adding concurrent image-based focus refinement.

In accordance with the techniques of the disclosure, image processor 112may apply an active-depth assisted focus with a concurrent image-basedfocus refinement (e.g., steps 804-808). For example, after imageprocessor 112 calculates the focus value (e.g., using contrastinformation and/or PD information) of images captured during a coarsesearch (e.g., step 806), image processor 112 may decide the nextmovement as shown in FIG. 8B. That is, rather than stepping back andforth for lens positions 854A-854D to identify a peak FV along FV curve857, techniques described herein may configure image processor 112 todetermine a direction to move camera lens 122 (e.g., only forward) thatcorresponds to an increase in FV along FV curve 857. In some examples,image processor 112 may determine a search range for moving camera lens122. In this way, image processor 112 may determine the peak FV along FVcurve 857 faster than techniques that step back and forth for lenspositions (e.g., for lens positions 854A-854D) even when fine search 812is not skipped.

For example, the step controller may step from lens position 856A tolens position 856B, to lens position 856C, and to lens position 856D.Step sizes between lens positions 856A-856D may be preconfigured. Insome examples, lens positions 856A-856D may form a range of lenspositions that includes a lens position corresponding to a focal pointfor the distance of an object. For example, the first lens position 856Amay be offset from the peak as determined using the object distance. Insome examples, the actual object distance would correspond to lensposition 856C.

FIG. 9A is a conceptual diagram showing an example of least squarepolynomial fitting process, in accordance with the techniques of thedisclosure. Lens positions 902A-902D (collectively, “lens positions902”) each correspond to a respective lens position (e.g., (1), (2),(3), (4)) of the focal curve 900. Curve 900 may represent a curvedetermined using a least square polynomial fitting process.

For example, image processor 112 may determine data associated with adistance between object 126 and device 100 using active-depth assistedfocus. In this example, image processor 112 may determine a plurality oflens positions of camera lens 122 based on the distance from object 126.For example, image processor 112 may determine lens position 902C as anestimated peak focus value mapped to the distance from object 126. Inthis example, image processor 112 may determine a second lens position902B by subtracting a preconfigured step size from the third lensposition and determine lens position 902A by subtracting thepreconfigured step size from the second lens position. Similarly, thestep controller may determine 902D by adding the preconfigured step sizeto 902C.

Image processor 112 may determine, for each one of lens positions 902, arespective focus value to generate a plurality of focus values. Forexample, image processor 112 may determine, for each one of lenspositions 902, phase difference information using PDAF techniques (e.g.,see FIGS. 12-13). In some examples, the image processor 112 maydetermine, for each one of lens positions 902, contrast information fora horizontal direction and/or contrast information for a verticaldirection using CDAF techniques.

FIG. 9B is a conceptual diagram showing an example details of a processfor active-depth assisted focus with concurrent image-based refinement,in accordance with the techniques of the disclosure. Image processor 112may use a curve fitting method (e.g., a least square polynomial curvefitting method) to obtain the peak of a polynomial as the best focusposition (e.g., a peak of curve 900 of FIG. 9A).

y=a _(k) x ^(k) + . . . +a ₁ x+a ₀+∈   EQUATION 4

After obtaining the best focus position, image processor 112 mayconsider one or more of the following to determine a best focusposition.

1. A ToF target 950 using, for example, an active-depth assisted focustarget (e.g., a distance to object)

2. A best focus position 952 (e.g., the fitting peak)

-   -   active-depth assisted focus target and the best focus position        should be within a reasonable range to ensure the correctness of        fitting peak.

3. The property of the polynomial curve 954 (e.g. shape, value, peak,etc.)

-   -   if the curve is not sharp enough, the fitting peak may not be        accurate.

Using one or more of the ToF target 950, best focus position 952, andproperty of the polynomial curve 954, image processor 112 may skip afine search 956 and may determine the final position 958 thatcorresponds to a selected lens position of camera lens 122. For example,in response to determining that the best focus position is withing areasonable range (e.g., within a threshold FV), image processor 112 mayskip fine search 956 and use final position 958 from best focus position952.

If a fine search cannot be skipped based on one or more of the ToFtarget 950, best focus position 952, and property of the polynomialcurve 954, image processor 112 may determine a configuration of a finesearch 960 and apply a fine search using the configuration 962. Forexample, image processor 112 may determine a direction (e.g., forwardsor backwards) to move camera lens 122 using one or more of the ToFtarget 950, best focus position 952, and property of the polynomialcurve 954. For instance, image processor 112 may determine a directionusing a current position and a search range. Image processor 112 maydetermine a search range based on peak FV values of contrast informationand/or PD information. In some examples, image processor 112 maydetermine a starting position for a fine search based on one or more ofa ToF target, a peak point (e.g., Hpeak and/or Vpeak), and PDinformation. For instance, image processor 112 may determine thestarting point, direction, and search range based on the ToF target, thepeak point, and the PD information.

FIG. 10 is a conceptual diagram showing two example cases for an examplefine search stage 1004 for active-depth assisted focus with concurrentimage-based refinement, in accordance with the techniques of thedisclosure. During image-based focus evaluator stage 1002, imageprocessor 112 may determine contrast information for a horizontaldirection. Contrast information for the horizontal direction may includea peak horizontal FV (also referred to herein as simply “Hpeak”). Asused herein, Hpeak refers to a focus value (FV) determined usingcontrast information in a horizontal direction. Hpeak may be alsoreferred to herein as contrast information for a horizontal direction.In some examples, contrast information for the horizontal direction mayinclude a confidence value, which may be referred to herein as simply“Hconf”). A higher Hconf may represent a more trustable Hpeak and alower Hconf may represent a less trustable Hpeak. Image processor 112may calculate Hconf based on a sharpness and a curve property of ahorizonal (‘H’) direction focus value curve.

During image-based focus evaluator stage 1002, image processor 112 maydetermine contrast information for a vertical direction. Contrastinformation for the vertical direction may include a peak vertical FV(also referred to herein as simply “Vpeak”). As used herein, Vpeakrefers to a focus value (FV) determined using contrast information inthe vertical direction. Vpeak may be also referred to herein as contrastinformation for a vertical direction. In some examples, contrastinformation for the vertical direction may include a confidence value,which may be referred to herein as simply “Vconf”). A higher Vconf valuemay represent a more trustable Vpeak and a lower Vconf may represent aless trustable Vpeak. Image processor 112 may calculate Vconf based on asharpness and a curve property of a vertical (V′) direction focus valuecurve.

During image-based focus evaluator stage 1002, image processor 112determine phase difference information. Phase difference information mayinclude a phase difference peak (also referred to herein as simply“PDpeak”). As used herein, PDpeak may be also referred to herein asphase difference information. In some examples, phase differenceinformation may include a confidence value, which may be referred toherein as simply “PDconf”). A higher PDconf may represent a moretrustable PDpeak and a lower PDconf may represent a less trustablePDpeak.

In the first case (Case 1), at least one of Hpeak, Vpeak, or PDpeak isnear the active-depth assisted focus target and the property ofpolynomial curve is qualified. In this example, the image would besufficiently in-focus and image processor 112 may skip the fine search1006 and determine the final position 1008. In the first case,techniques described herein may use more image information to cover avarious scene, weighting the three peaks as a final best peak. Forexample, when fine search is skipped, image processor 112 may determinethe best peak FV as the combination of Hpeak, Vpeak, and PDpeak withdifferent weighting. The weighing may be determined by the confidenceand flexible tuning. That is, image processor 112 may skip fine searchand determine the final position for camera lens 122 based on contrastinformation (e.g., Hpeak, Vpeak) and/or phase difference information(e.g., PD peak). For instance, image processor 112 may determine thefinal position for a lens of a camera using equation 5.

FINAL BEST PEAK=WEIGHT1*H _(peak)+WEIGHT2*V _(peak) WEIGHT3*PD_(peak)  EQUATION 5

where FINAL BEST PEAK is determined FV peak value, WEIGHT1 is a firstweight value, WEIGHT2 is a second weight value, and WEIGHT3 is a thirdweight value. In some examples, one or more of WEIGHT1, WEIGHT2, orWEIGHT3 may be zero. For example, fine search controller 984 maydetermine the final best peak based on PDpeak (e.g., WEIGHT3 isnon-zero) and may set WEIGHT1 and WEIGHT2 to zero.

In a second case (Case 2), none of the Hpeak, Vpeak, and PDpeak is nearthe TOF target. In this example, image processor 112 may use therelationship between the TOF target and the FV peak of each to determineconfiguration of a fine search 1010 and apply the fine search using theconfiguration (1012). For example, image processor 112 may set a startpoint (e.g., a best confidence peak in the Hpeak, Vpeak, and PDpeak) anda search range (if Hpeak=90 Vpeak=100 PDpeak=120 the search range willcover 90˜120). In this way, image processor 112 may perform the finesearch with fewer steps compared to systems that do not use the startpoint.

Image processor 112 may determine the search range using the peak ofHpeak, Vpeak, and PDpeak. For example, image processor 112 may set thesearch range to at least cover the peak of Hpeak, Vpeak, and PDpeak. Forinstance, if Hpeak=90 Vpeak=100 PDpeak=120, image processor 112 maydetermine the search range to at least cover 90˜120.

Image processor 112 may determine the direction of a fine search using acurrent position and a search range. For example, if current lensposition is 80 and search range is 90˜120, fine search controller 984may determine the direction to be from 90 to 120 (e.g., in a positivedirection). In some examples, if the current lens position is 130 andthe search range is 90˜120, image processor 112 may determine thedirection to be from 90 to 120 (e.g., in a negative or backwardsdirection).

FIG. 11 is a conceptual diagram showing an example process foractive-depth assisted focus with concurrent image-based refinement thatdecides whether to skip a fine search, in accordance with the techniquesof the disclosure. Using the one or more of the Hpeak, Vpeak, or PDpeak,image processor 112 may determine one or more of the following.

1. Whether to skip a fine search

2. A final lens position of the lens

3. The configuration of fine search if the fine search is not skipped.

-   -   the configuration may include a direction, a start point, and a        number of steps of the fine search.

For example, image processor 112 may determine a final lens position ofcamera lens 122 based on a plurality of focus values. For instance,image processor 112 may determine whether the respective focus value fora particular lens position of the plurality of lens positions is withina threshold range from a peak focus value (e.g., Hpeak, Vpeak, orPDpeak) for the distance from the object (1102). In some instances,image processor 112 may determine whether the respective focus value(e.g., Hpeak, Vpeak, or PDpeak) for the particular lens positioncorresponds to an estimated local maximum of focus values.

In response a determination that the respective focus value for theparticular lens position is within the threshold range from the peakfocus value for the distance of the object and a determination that therespective focus value for the particular lens position corresponds tothe estimated local maximum of focus values, image processor 112 mayskip a fine search (1104). In this example, image processor 112 mayoutput instructions to move camera lens 122 to the final lens positionwithout performing the fine search (1106).

However, based on a determination that the respective focus value forthe particular lens position is not within the threshold range from thepeak focus value for the distance of the object and a determination thatthe respective focus value for the particular lens position does notcorrespond to the estimated local maximum of focus values, imageprocessor 112 may perform the fine search based on the plurality offocus values (1108).

Image processor 112 may decide a configuration for the fine search(1110). For example, image processor 112 may determine a start point forthe fine search based on the plurality of lens position and theplurality of focus values. In some examples, image processor 112 maydetermine a direction for the fine search based on the plurality of lensposition and the plurality of focus values. In some examples, imageprocessor 112 may determine a search range for the fine search based onthe plurality of lens position and the plurality of focus values. Imageprocessor 112 may perform the fine search to determine the final lensposition of camera lens 122 and output instructions to move camera lens122 to the final lens position determined using the fine search.

With the active-depth assisted focus techniques described herein usingactive-depth assisted focus with concurrent image-based refinement, auser experience relating to a photography of people may be improved andan accuracy of the resulting image even on very low light condition maybe improved compared to systems using CDAF, active-depth assisted focuswith a fine search, and PDAF.

FIG. 12 is a conceptual diagram of an example phase detection auto focus(PDAF) sensor. Image processor 112 may, using the process of FIG. 12,generate phase difference information from images acquired at the sametime as the ToF measurements, which may be used by image processor 112to perform image-based focus refinement using the phase differenceinformation. In the example of FIG. 12, camera 102 includes a PDAFsensor array 70. PDAF sensor array 70 may generate phase differenceinformation from images acquired at the same time as the ToFmeasurements, which may be used by device 100 to perform image-basedfocus refinement using the phase difference information. Image sensor124 may include PDAF sensor array 70.

PDAF sensor array 70 shows an 8×8 portion of an example sensor array forpurposes of illustration. Some sensor arrays may include millions ofpixels. Each pixel in PDAF sensor array 70 may be divided into a leftand right pixel, as shown by the dashed line. In addition, each pixel inPDAF sensor array is associated with a color filter. The pattern of thecolor filters shown in FIG. 12 is a called a Bayer filter. A Bayerfilter is a color filter array for Red (R), Blue (B), and Green (G)filters arranged on the pixels (e.g., photodiodes or photosensors) ofPDAF sensor array 70.

The Bayer filter results in each pixel only capturing one color valuefor each pixel. Image processor 112 may produce a full-color image byperforming a demosaicing algorithm to interpolate a complete RGB valuefor each pixel. Demosaicing algorithms use the various color values ofsurrounding pixels to produce a full color value for a particular pixel.

FIG. 12 also shows an enlarged view of a single pixel 72. Pixel 72, likethe other pixels in PDAF sensor array 70, is configured as a phasedetection pixel. In this example, pixel 72 is configured as a dualphotodiode. However, the phase detection pixels of sensor array 70 maybe configured in any manner to receive and output phase difference pixelvalues.

Pixel 72 may include a mask 82 that divides the photodiodes of pixel 72into a left photodiode 78 and a right photodiode 80. Pixel 72 need notbe limited to photodiodes, but may use any type of photosensors that maybe configured to receive light and output an electrical signalindicating the amount of light received. Pixel 72 may further include agreen color filter 76 and an on-chip lens 74. Of course, given theposition of pixel 72 in the Bayer filter mask of PDAF sensor array 70,the color filter may be a different color (e.g., red or blue).

Light passes through on-chip lens 74 and then through green color filter76. The intensity of the green light that is then incident on leftphotodiode 78 and right photodiode 78 is then converted into anelectrical signal. The electrical signal may then be converted to adigital value (e.g., using an analog-to-digital converter). Note thaton-chip lens 74 is not the same as the main lens of camera 102. On-chiplens 74 is a fixed lens associated with pixel 72 of PDAF sensor array70, while the main lens is the adjustable lens that is part of theentire package of camera 102.

FIG. 13 is a conceptual diagram showing one example technique fordetermining phase difference pixel data in a PDAF sensor. In the exampleFIG. 12, camera 102 is a type 2 PDAF sensor (labeled as camera 102A).PDAF sensor array 70 may generate phase difference information fromimages acquired at the same time as the ToF measurements, which may beused by device 100 to perform image-based focus refinement using thephase difference information. Type2 PDAF sensors (e.g., camera 102A) area commonly used sensor type. Examples include the SS2P7 sensor made bySamsung Electronics of Yeongtown, South Korea, and the IMX362 sensormade by the Sony Corporation of Tokyo, Japan.

As described above, in a type2 PDAF sensor, the separation and phasedifference correction on the pixel data stream may be performed insidethe sensor. After separation and correction, the pixel data received bycamera 102A may be averaged and the pixel data is transferred to imageprocessor 112. For example, image processor 112 may be configured toaverage both the value of the right photodiode (Rp) and the leftphotodiode (Lp) to form a complete pixel, which is transferred to imageprocessor 112 with a unique data type. The data type may indicate toimage processor 112 that the data is pixel data that is to be used toprocess and receive an image (e.g., that the data is not to be used forauto focus determination).

In a type2 PDAF sensor, the phase calculation process may be performedby image processor 112. In some examples, a processor at camera 102 maybe configured to send an entire stream of phase difference pixel data(e.g., phase difference values for every phase detection pixel in thearray of camera 102A) to image processor 112. Image processor 112 mayperform the phase difference calculation to obtain the defocus valuesand translate them into a lens position. Image processor 112 maycalculate the phase difference pixel data using interpolation of thevalues of neighbor phase detection pixels.

FIG. 13 shows one example of how to calculate phase difference valuesfor a particular phase detection pixel. Image processor 112 maycalculate the phase difference pixel data for both the left photodiode(designated YL) and the right photodiode (designated YR). Imageprocessor 112 may generate phase difference information from imagesacquired at the same time as the ToF measurements, which may be used toperform image-based focus refinement using the phase differenceinformation.

For example, for pixel 84, image processor 112 may use the surroundingpixel data values in region 86 to interpolate the phase differencevalues. Image processor 112 may interpolate the phase difference valuefor the left photodiode of pixel 84 (YL) and the right photodiode ofpixel 84 (YR) as follows:

Y _(L) =a*R _(L) +b*G _(L) +c*B _(L)   EQUATION 6

Y _(R) =a′*R _(R) +b′*G _(R) +c′*B _(R)   EQUATION 7

Image processor 112 may determine the values of R_(L), R_(R), G_(L),G_(R), B_(L), and B_(R) from the photo detection pixels of region 86.The values of constants a, a′, b, b′, c, and c′ may be the same as thoseused in RGB to YUV or YCrCb color space conversions. However, the valuesof the constants may be changed based on user preferences. Exampledefault values may be a=0.299, b=0.587, and c=0.114. FIG. 13 shows oneexample of how image processor 112 may be configured to calculate phasedifference values. However, other techniques may be used.

FIG. 14 is a flowchart illustrating an example method of operationaccording to one or more example techniques described in thisdisclosure. FIG. 14 is described using image processor 112 of FIG. 1 forexample purposes only. Image processor 112 may determine data associatedwith a distance between object 126 and device 100 (1402). For example,image processor 112 may apply a process for laser autofocus and/ortime-of-flight to determine the data associated with the distancebetween object 126 and device 100. Image processor 112 may determine aplurality of lens positions of camera lens 122 based on the dataassociated with the distance between object 126 and device 100 (1404).For example, image processor 112 may determine lens positions 902A-902Dof FIG. 9A based on the distance of image sensor 124 from object 126.Image processor 112 may cause camera lens 122 to move to each of theplurality of lens positions in response to determining the plurality oflens positions.

Image processor 112 may determine, for each one of the plurality of lenspositions, a respective focus value to generate a plurality of focusvalues (1406). To determine, for each one of the plurality of lenspositions, the respective focus value, image processor 112 maydetermine, for each one of the plurality of lens positions, phasedifference information. For example, image processor 112 may determine afocus value for each one of lens positions 902A-902D of FIG. 9A using aphase difference. Determining a focus value for each one of a pluralityof lens positions using a phase difference may improve an accuracy ofthe focus values compared to systems relying only on CDAF. Improving anaccuracy of the focus values may increase a speed of AF, which mayimprove a user experience relating to a photography compared to systemsusing CDAF, active-depth assisted focus with a fine search, and PDAF.

Image processor 112 may determine a final lens position based on theplurality of focus values (1408). For example, image processor 112 maydetermine whether the respective focus value for a particular lensposition of a plurality of lens positions (e.g., lens positions902A-902D) is within a threshold range from a peak focus value for thedata associated with the distance between object 126 and device 100. Insome examples, image processor 112 may determine whether the respectivefocus value for the particular lens position corresponds to an estimatedlocal maximum of focus values.

Image processor 112 may, in response to a determination that therespective focus value for the particular lens position of camera lens122 is within the threshold range from the peak focus value for thedistance of the object and a determination that the respective focusvalue for the particular lens position of camera lens 122 corresponds tothe estimated local maximum of focus values, skip a fine search. In thisexample, image processor 112 may select a lens position from theplurality of lens positions with a highest focus value. Skipping a finesearch may increase a speed of AF, which may improve a user experiencerelating to a photography compared to systems using CDAF, active-depthassisted focus with a fine search, and PDAF.

In some examples, image processor 112 may be configured to perform afine search based on a determination that the respective focus value forthe particular lens position is not within the threshold range from thepeak focus value for the distance of the object and a determination thatthe respective focus value for the particular lens position does notcorrespond to the estimated local maximum of focus values. For example,image processor 112 may determine a start point for the fine searchbased on the plurality of lens position and the plurality of focusvalues. In some examples, image processor 112 may determine a directionfor the fine search based on the plurality of lens position and theplurality of focus values. In some examples, image processor 112 maydetermine a search range for the fine search based on the plurality oflens position and the plurality of focus values.

Image processor 112 may perform a fine search using one or more of thestart point, direction, or search range. Performing a fine search usingone or more of the start point, direction, or search range may improve aspeed of AF, which may improve a user experience relating to aphotography compared to systems using CDAF, active-depth assisted focuswith a fine search, and PDAF.

Image processor 112 may output instructions to move the camera lens tothe final lens position (1410). Image processor 112 may position, withcamera 102, camera lens 122 to the final lens position. Camera 102 maycapture an image with the camera lens positioned to the final lensposition (1412). One or more processors 110 may output the image atdisplay 114 (1414).

In one or more examples, the functions described may be implemented inhardware, software, firmware, or any combination thereof. If implementedin software, the functions may be stored on or transmitted over, as oneor more instructions or code, a computer-readable medium and executed bya hardware-based processing unit. Computer-readable media may includecomputer-readable storage media, which corresponds to a tangible mediumsuch as data storage media. In this manner, computer-readable mediagenerally may correspond to tangible computer-readable storage mediawhich is non-transitory. Data storage media may be any available mediathat can be accessed by one or more computers or one or more processorsto retrieve instructions, code and/or data structures for implementationof the techniques described in this disclosure. A computer programproduct may include a computer-readable medium.

By way of example, and not limitation, such computer-readable storagemedia can comprise RAM, ROM, EEPROM, CD-ROM or other optical diskstorage, magnetic disk storage, or other magnetic storage devices, flashmemory, cache memory, or any other medium that can be used to storedesired program code in the form of instructions or data structures andthat can be accessed by a computer. It should be understood thatcomputer-readable storage media and data storage media do not includecarrier waves, signals, or other transient media, but are insteaddirected to non-transient, tangible storage media. Disk and disc, asused herein, includes compact disc (CD), laser disc, optical disc,digital versatile disc (DVD), floppy disk and Blu-ray disc, where discsusually reproduce data magnetically, while discs reproduce dataoptically with lasers. Combinations of the above should also be includedwithin the scope of computer-readable media.

Instructions may be executed by one or more processors, such as one ormore digital signal processors (DSPs), general purpose microprocessors,application specific integrated circuits (ASICs), field programmablelogic arrays (FPGAs), or other equivalent integrated or discrete logiccircuitry. Accordingly, the term “processor,” as used herein may referto any of the foregoing structure or any other structure suitable forimplementation of the techniques described herein. In addition, in someaspects, the functionality described herein may be provided withindedicated hardware and/or software modules configured for encoding anddecoding, or incorporated in a combined codec. Also, the techniquescould be fully implemented in one or more circuits or logic elements.

The techniques of this disclosure may be implemented in a wide varietyof devices or apparatuses, including a wireless handset, an integratedcircuit (IC) or a set of ICs (e.g., a chip set). Various components,modules, or units are described in this disclosure to emphasizefunctional aspects of devices configured to perform the disclosedtechniques, but do not necessarily require realization by differenthardware units. Rather, as described above, various units may becombined in a codec hardware unit or provided by a collection ofinteroperative hardware units, including one or more processors asdescribed above, in conjunction with suitable software and/or firmware.

The following clauses are a non-limiting list of clauses in accordancewith one or more techniques of this disclosure.

Clause A1. An apparatus configured for image processing, the apparatuscomprising a processor configured to: determine a distance from anobject; determine a plurality of lens positions of a camera lens basedon the distance from the object; determine, for each one of theplurality of lens positions, a respective focus value to generate aplurality of focus values, wherein, to determine, for each one of theplurality of lens positions, the respective focus value, the processoris configured to determine, for each one of the plurality of lenspositions, phase difference information; and determine a final lensposition based on the plurality of focus values.

Clause A2. The apparatus of clause A1, wherein the processor is furtherconfigured to output instructions to move the camera lens to the finallens position.

Clause A3. The apparatus of one or more of clauses A1-A2, wherein, todetermine, for each one of the plurality of lens positions, therespective focus value, the processor is further configured todetermine, for each one of the plurality of lens positions, one or moreof: contrast information for a horizontal direction; or contrastinformation for a vertical direction.

Clause A4. The apparatus of one or more of clauses A1-A3, wherein, todetermine the final lens position, the processor is configured to:determine whether the respective focus value for a particular lensposition of the plurality of lens positions is within a thresholddistance from a peak focus value for the distance from the object;and/or determine whether the respective focus value for the particularlens position corresponds to a local maximum of focus values.

Clause A5. The apparatus of clause A4, wherein, to determine the finallens position, the processor is configured to, in response to adetermination that the respective focus value for the particular lensposition is within the threshold distance from the peak focus value forthe distance of the object and a determination that the respective focusvalue for the particular lens position corresponds to the local maximumof focus values, skip a fine search.

Clause A6. The apparatus of clause A4, wherein, to determine the finallens position, the processor is configured to perform a fine searchbased on the plurality of focus values based on a determination that therespective focus value for the particular lens position is not withinthe threshold distance from the peak focus value for the distance of theobject and a determination that the respective focus value for theparticular lens position does not correspond to the local maximum offocus values.

Clause A7. The apparatus of clause A6, wherein, to determine the finallens position, the processor is configured to: determine a start pointfor the fine search based on the plurality of lens position and theplurality of focus values; determine a direction for the fine searchbased on the plurality of lens position and the plurality of focusvalues; and/or determine a search range for the fine search based on theplurality of lens position and the plurality of focus values.

Clause A8. The apparatus of one or more of clauses A1-A7, wherein, todetermine the distance of the object, the processor is configured toperform an active-depth assisted focus operation.

Clause B1. An apparatus configured for image processing, the apparatuscomprising one or more processors implemented in circuitry andconfigured to: determine data associated with a distance between anobject and the apparatus; determine a plurality of lens positions of acamera lens based on the data associated with the distance between theobject and the apparatus; determine, for each one of the plurality oflens positions, a respective focus value to generate a plurality offocus values, wherein, to determine, for each one of the plurality oflens positions, the respective focus value, the one or more processorsare configured to determine, for each one of the plurality of lenspositions, phase difference information; and determine a final lensposition based on the plurality of focus values.

Clause B2. The apparatus of clause B1, wherein the one or moreprocessors are further configured to output instructions to move thecamera lens to the final lens position.

Clause B3. The apparatus of clause B1, wherein, to determine, for eachone of the plurality of lens positions, the respective focus value, theone or more processors are further configured to determine, for each oneof the plurality of lens positions, one or more of: contrast informationfor a horizontal direction; or contrast information for a verticaldirection.

Clause B4. The apparatus of clause B1, wherein, to determine the finallens position, the one or more processors are configured to: determinewhether the respective focus value for a particular lens position of theplurality of lens positions is within a threshold range from a peakfocus value for the data associated with the distance between the objectand the apparatus; and determine whether the respective focus value forthe particular lens position corresponds to an estimated local maximumof focus values.

Clause B5. The apparatus of clause B4, wherein, to determine the finallens position, the one or more processors are configured to skip a finesearch in response to a determination that the respective focus valuefor the particular lens position is within the threshold range from thepeak focus value for the distance of the object and a determination thatthe respective focus value for the particular lens position correspondsto the estimated local maximum of focus values.

Clause B6. The apparatus of clause B4, wherein, to determine the finallens position, the one or more processors are configured to perform afine search based on the plurality of focus values based on adetermination that the respective focus value for the particular lensposition is not within the threshold range from the peak focus value forthe distance of the object and a determination that the respective focusvalue for the particular lens position does not correspond to theestimated local maximum of focus values.

Clause B7. The apparatus of clause B6, wherein, to determine the finallens position, the one or more processors are configured to: determine astart point for the fine search based on the plurality of lens positionand the plurality of focus values; determine a direction for the finesearch based on the plurality of lens position and the plurality offocus values; or determine a search range for the fine search based onthe plurality of lens position and the plurality of focus values.

Clause B8. The apparatus of clause B1, wherein, to determine thedistance of the object, the one or more processors are configured toperform an active-depth assisted focus operation.

Clause B9. The apparatus of clause B1, wherein, to determine thedistance of the object, the one or more processors are configured toperform a course search.

Clause B10. The apparatus of clause B1, wherein, to determine thedistance of the object, the one or more processors are configured toperform phase detection auto focus (PDAF).

Clause B11. The apparatus of clause B1, further comprising a cameraconfigured to: position the camera lens to the final lens position; andcapture an image with the camera lens positioned at the final lensposition.

Clause B12. The apparatus of clause B11, further comprising a displayconfigured to output the image.

Clause B13. The apparatus of clause B1, wherein the one or moreprocessors are configured to cause the camera lens to move to each ofthe plurality of lens positions in response to determining the pluralityof lens positions.

Clause B14. A method for image processing, the method comprising:determining, with one or more processors implemented in circuitry, dataassociated with a distance between an object and a camera lens;determining, with the one or more processors, a plurality of lenspositions of the camera lens based on the data associated with thedistance between the object and the camera lens; determining, with theone or more processors and for each one of the plurality of lenspositions, a respective focus value to generate a plurality of focusvalues, wherein determining, for each one of the plurality of lenspositions, the respective focus value comprises determining, for eachone of the plurality of lens positions, phase difference information;and determining, with the one or more processors, a final lens positionbased on the plurality of focus values.

Clause B15. The method of clause B14, further comprising outputting,with the one or more processors, instructions to move the camera lens tothe final lens position.

Clause B16. The method of clause B14, wherein determining, for each oneof the plurality of lens positions, the respective focus value,comprises determining, for each one of the plurality of lens positions,one or more of: contrast information for a horizontal direction; orcontrast information for a vertical direction.

Clause B17. The method of clause B14, wherein determining the final lensposition comprises: determining whether the respective focus value for aparticular lens position of the plurality of lens positions is within athreshold range from a peak focus value for the data associated with thedistance between the object and the camera lens; and determining whetherthe respective focus value for the particular lens position correspondsto an estimated local maximum of focus values.

Clause B18. The method of clause B17, wherein determining the final lensposition comprises skipping a fine search in response to a determinationthat the respective focus value for the particular lens position iswithin the threshold range from the peak focus value for the distance ofthe object and a determination that the respective focus value for theparticular lens position corresponds to the estimated local maximum offocus values.

Clause B19. The method of clause B17, wherein determining the final lensposition comprises performing a fine search based on the plurality offocus values based on a determination that the respective focus valuefor the particular lens position is not within the threshold range fromthe peak focus value for the distance of the object and a determinationthat the respective focus value for the particular lens position doesnot correspond to the estimated local maximum of focus values.

Clause B20. The method of clause B19, wherein determining the final lensposition comprises: determining a start point for the fine search basedon the plurality of lens position and the plurality of focus values;determining a direction for the fine search based on the plurality oflens position and the plurality of focus values; or determining a searchrange for the fine search based on the plurality of lens position andthe plurality of focus values.

Clause B21. The method of clause B14, wherein determining the distanceof the object comprises performing an active-depth assisted focusoperation.

Clause B22. The method of clause B14, wherein determining the distanceof the object comprises performing a course search.

Clause B23. The method of clause B14, wherein determining the distanceof the object comprises performing phase detection auto focus (PDAF).

Clause B24. The method of clause B14, further comprising: positioning,with the one or more processors and with a camera, the camera lens tothe final lens position; and capturing, with the one or more processorsand with the camera, an image with the camera lens positioned at thefinal lens position.

Clause B25. The method of clause B24, further comprising outputting,with the one or more processors, the image at a display.

Clause B26. The method of clause B14, further comprising causing, withthe one or more processors, the camera lens to move to each of theplurality of lens positions in response to determining the plurality oflens positions.

Clause B27. A device for image processing, the device comprising: meansfor determining data associated with a distance between an object andthe device; means for determining a plurality of lens positions of acamera lens based on the data associated with the distance between theobject and the device; means for determining, for each one of theplurality of lens positions, a respective focus value to generate aplurality of focus values, wherein the means for determining, for eachone of the plurality of lens positions, the respective focus valuecomprises means for determining, for each one of the plurality of lenspositions, phase difference information; and means for determining afinal lens position based on the plurality of focus values.

Clause B28. A computer-readable storage medium having stored thereoninstructions that, when executed, configure one or more processors to:determine data associated with a distance between an object and a cameralens; determine a plurality of lens positions of the camera lens basedon the data associated with the distance between the object and thecamera lens; determine, for each one of the plurality of lens positions,a respective focus value to generate a plurality of focus values,wherein the instructions that cause the one or more processors todetermine, for each one of the plurality of lens positions, therespective focus value further cause the processor to determine, foreach one of the plurality of lens positions, phase differenceinformation; and determine a final lens position based on the pluralityof focus values.

Clause C1. An apparatus configured for image processing, the apparatuscomprising one or more processors implemented in circuitry andconfigured to: determine data associated with a distance between anobject and the apparatus; determine a plurality of lens positions of acamera lens based on the data associated with the distance between theobject and the apparatus; determine, for each one of the plurality oflens positions, a respective focus value to generate a plurality offocus values, wherein, to determine, for each one of the plurality oflens positions, the respective focus value, the one or more processorsare configured to determine, for each one of the plurality of lenspositions, phase difference information; and determine a final lensposition based on the plurality of focus values.

Clause C2. The apparatus of clause C1, wherein the one or moreprocessors are further configured to output instructions to move thecamera lens to the final lens position.

Clause C3. The apparatus of any combination of clauses C1-C2, wherein,to determine, for each one of the plurality of lens positions, therespective focus value, the one or more processors are furtherconfigured to determine, for each one of the plurality of lenspositions, one or more of: contrast information for a horizontaldirection; or contrast information for a vertical direction.

Clause C4. The apparatus of any combination of clauses C1-C3, wherein,to determine the final lens position, the one or more processors areconfigured to: determine whether the respective focus value for aparticular lens position of the plurality of lens positions is within athreshold range from a peak focus value for the data associated with thedistance between the object and the apparatus; and determine whether therespective focus value for the particular lens position corresponds toan estimated local maximum of focus values.

Clause C5. The apparatus of clause C4, wherein, to determine the finallens position, the one or more processors are configured to skip a finesearch in response to a determination that the respective focus valuefor the particular lens position is within the threshold range from thepeak focus value for the distance of the object and a determination thatthe respective focus value for the particular lens position correspondsto the estimated local maximum of focus values.

Clause C6. The apparatus of clause C4, wherein, to determine the finallens position, the one or more processors are configured to perform afine search based on the plurality of focus values based on adetermination that the respective focus value for the particular lensposition is not within the threshold range from the peak focus value forthe distance of the object and a determination that the respective focusvalue for the particular lens position does not correspond to theestimated local maximum of focus values.

Clause C7. The apparatus of clause C6, wherein, to determine the finallens position, the one or more processors are configured to: determine astart point for the fine search based on the plurality of lens positionand the plurality of focus values; determine a direction for the finesearch based on the plurality of lens position and the plurality offocus values; or determine a search range for the fine search based onthe plurality of lens position and the plurality of focus values.

Clause C8. The apparatus of any combination of clauses C1-C7, wherein,to determine the distance of the object, the one or more processors areconfigured to perform an active-depth assisted focus operation.

Clause C9. The apparatus of any combination of clauses C1-C8, wherein,to determine the distance of the object, the one or more processors areconfigured to perform a course search.

Clause C10. The apparatus of any combination of clauses C1-C8, wherein,to determine the distance of the object, the one or more processors areconfigured to perform phase detection auto focus (PDAF).

Clause C11. The apparatus of any combination of clauses C1-C10, furthercomprising a camera configured to: position the camera lens to the finallens position; and capture an image with the camera lens positioned atthe final lens position.

Clause C12. The apparatus of clause C11, further comprising a displayconfigured to output the image.

Clause C13. The apparatus of any combination of clauses C1-C12, whereinthe one or more processors are configured to cause the camera lens tomove to each of the plurality of lens positions in response todetermining the plurality of lens positions.

Clause C14. A method for image processing, the method comprising:determining, with one or more processors implemented in circuitry, dataassociated with a distance between an object and a camera lens;determining, with the one or more processors, a plurality of lenspositions of the camera lens based on the data associated with thedistance between the object and the camera lens; determining, with theone or more processors and for each one of the plurality of lenspositions, a respective focus value to generate a plurality of focusvalues, wherein determining, for each one of the plurality of lenspositions, the respective focus value comprises determining, for eachone of the plurality of lens positions, phase difference information;and determining, with the one or more processors, a final lens positionbased on the plurality of focus values.

Clause C15. The method of clause C14, further comprising outputting,with the one or more processors, instructions to move the camera lens tothe final lens position.

Clause C16. The method of any combination of clauses C14-C15, whereindetermining, for each one of the plurality of lens positions, therespective focus value, comprises determining, for each one of theplurality of lens positions, one or more of: contrast information for ahorizontal direction; or contrast information for a vertical direction.

Clause C17. The method of any combination of clauses C14-C16, whereindetermining the final lens position comprises: determining whether therespective focus value for a particular lens position of the pluralityof lens positions is within a threshold range from a peak focus valuefor the data associated with the distance between the object and thecamera lens; and determining whether the respective focus value for theparticular lens position corresponds to an estimated local maximum offocus values.

Clause C18. The method of clause C17, wherein determining the final lensposition comprises skipping a fine search in response to a determinationthat the respective focus value for the particular lens position iswithin the threshold range from the peak focus value for the distance ofthe object and a determination that the respective focus value for theparticular lens position corresponds to the estimated local maximum offocus values.

Clause C19. The method of clause C17, wherein determining the final lensposition comprises performing a fine search based on the plurality offocus values based on a determination that the respective focus valuefor the particular lens position is not within the threshold range fromthe peak focus value for the distance of the object and a determinationthat the respective focus value for the particular lens position doesnot correspond to the estimated local maximum of focus values.

Clause C20. The method of clause C19, wherein determining the final lensposition comprises: determining a start point for the fine search basedon the plurality of lens position and the plurality of focus values;determining a direction for the fine search based on the plurality oflens position and the plurality of focus values; or determining a searchrange for the fine search based on the plurality of lens position andthe plurality of focus values.

Clause C21. The method of any combination of clauses C14-C20, whereindetermining the distance of the object comprises performing anactive-depth assisted focus operation.

Clause C22. The method of any combination of clauses C14-C21, whereindetermining the distance of the object comprises performing a coursesearch.

Clause C23. The method of any combination of clauses C14-C22, whereindetermining the distance of the object comprises performing phasedetection auto focus (PDAF).

Clause C24. The method of any combination of clauses C14-C23, furthercomprising: positioning, with the one or more processors and with acamera, the camera lens to the final lens position; and capturing, withthe one or more processors and with the camera, an image with the cameralens positioned at the final lens position.

Clause C25. The method of clause C24, further comprising outputting,with the one or more processors, the image at a display.

Clause C26. The method of any combination of clauses C14-C25, furthercomprising causing, with the one or more processors, the camera lens tomove to each of the plurality of lens positions in response todetermining the plurality of lens positions.

Clause C27. A device for image processing, the device comprising: meansfor determining data associated with a distance between an object andthe device; means for determining a plurality of lens positions of acamera lens based on the data associated with the distance between theobject and the device; means for determining, for each one of theplurality of lens positions, a respective focus value to generate aplurality of focus values, wherein the means for determining, for eachone of the plurality of lens positions, the respective focus valuecomprises means for determining, for each one of the plurality of lenspositions, phase difference information; and means for determining afinal lens position based on the plurality of focus values.

Clause C28. A computer-readable storage medium having stored thereoninstructions that, when executed, configure one or more processors to:determine data associated with a distance between an object and a cameralens; determine a plurality of lens positions of the camera lens basedon the data associated with the distance between the object and thecamera lens; determine, for each one of the plurality of lens positions,a respective focus value to generate a plurality of focus values,wherein the instructions that cause the one or more processors todetermine, for each one of the plurality of lens positions, therespective focus value further cause the processor to determine, foreach one of the plurality of lens positions, phase differenceinformation; and determine a final lens position based on the pluralityof focus values.

Various examples have been described. These and other examples arewithin the scope of the following claims.

What is claimed is:
 1. An apparatus configured for image processing, theapparatus comprising one or more processors implemented in circuitry andconfigured to: determine data associated with a distance between anobject and the apparatus; determine a plurality of lens positions of acamera lens based on the data associated with the distance between theobject and the apparatus; determine, for each one of the plurality oflens positions, a respective focus value to generate a plurality offocus values, wherein, to determine, for each one of the plurality oflens positions, the respective focus value, the one or more processorsare configured to determine, for each one of the plurality of lenspositions, phase difference information; and determine a final lensposition based on the plurality of focus values.
 2. The apparatus ofclaim 1, wherein the one or more processors are further configured tooutput instructions to move the camera lens to the final lens position.3. The apparatus of claim 1, wherein, to determine, for each one of theplurality of lens positions, the respective focus value, the one or moreprocessors are further configured to determine, for each one of theplurality of lens positions, one or more of: contrast information for ahorizontal direction; or contrast information for a vertical direction.4. The apparatus of claim 1, wherein, to determine the final lensposition, the one or more processors are configured to: determinewhether the respective focus value for a particular lens position of theplurality of lens positions is within a threshold range from a peakfocus value for the data associated with the distance between the objectand the apparatus; and determine whether the respective focus value forthe particular lens position corresponds to an estimated local maximumof focus values.
 5. The apparatus of claim 4, wherein, to determine thefinal lens position, the one or more processors are configured to skip afine search in response to a determination that the respective focusvalue for the particular lens position is within the threshold rangefrom the peak focus value for the distance of the object and adetermination that the respective focus value for the particular lensposition corresponds to the estimated local maximum of focus values. 6.The apparatus of claim 4, wherein, to determine the final lens position,the one or more processors are configured to perform a fine search basedon the plurality of focus values based on a determination that therespective focus value for the particular lens position is not withinthe threshold range from the peak focus value for the distance of theobject and a determination that the respective focus value for theparticular lens position does not correspond to the estimated localmaximum of focus values.
 7. The apparatus of claim 6, wherein, todetermine the final lens position, the one or more processors areconfigured to: determine a start point for the fine search based on theplurality of lens position and the plurality of focus values; determinea direction for the fine search based on the plurality of lens positionand the plurality of focus values; or determine a search range for thefine search based on the plurality of lens position and the plurality offocus values.
 8. The apparatus of claim 1, wherein, to determine thedistance of the object, the one or more processors are configured toperform an active-depth assisted focus operation.
 9. The apparatus ofclaim 1, wherein, to determine the distance of the object, the one ormore processors are configured to perform a course search.
 10. Theapparatus of claim 1, wherein, to determine the distance of the object,the one or more processors are configured to perform phase detectionauto focus (PDAF).
 11. The apparatus of claim 1, further comprising acamera configured to: position the camera lens to the final lensposition; and capture an image with the camera lens positioned at thefinal lens position.
 12. The apparatus of claim 11, further comprising adisplay configured to output the image.
 13. The apparatus of claim 1,wherein the one or more processors are configured to cause the cameralens to move to each of the plurality of lens positions in response todetermining the plurality of lens positions.
 14. A method for imageprocessing, the method comprising: determining, with one or moreprocessors implemented in circuitry, data associated with a distancebetween an object and a camera lens; determining, with the one or moreprocessors, a plurality of lens positions of the camera lens based onthe data associated with the distance between the object and the cameralens; determining, with the one or more processors and for each one ofthe plurality of lens positions, a respective focus value to generate aplurality of focus values, wherein determining, for each one of theplurality of lens positions, the respective focus value comprisesdetermining, for each one of the plurality of lens positions, phasedifference information; and determining, with the one or moreprocessors, a final lens position based on the plurality of focusvalues.
 15. The method of claim 14, further comprising outputting, withthe one or more processors, instructions to move the camera lens to thefinal lens position.
 16. The method of claim 14, wherein determining,for each one of the plurality of lens positions, the respective focusvalue, comprises determining, for each one of the plurality of lenspositions, one or more of: contrast information for a horizontaldirection; or contrast information for a vertical direction.
 17. Themethod of claim 14, wherein determining the final lens positioncomprises: determining whether the respective focus value for aparticular lens position of the plurality of lens positions is within athreshold range from a peak focus value for the data associated with thedistance between the object and the camera lens; and determining whetherthe respective focus value for the particular lens position correspondsto an estimated local maximum of focus values.
 18. The method of claim17, wherein determining the final lens position comprises skipping afine search in response to a determination that the respective focusvalue for the particular lens position is within the threshold rangefrom the peak focus value for the distance of the object and adetermination that the respective focus value for the particular lensposition corresponds to the estimated local maximum of focus values. 19.The method of claim 17, wherein determining the final lens positioncomprises performing a fine search based on the plurality of focusvalues based on a determination that the respective focus value for theparticular lens position is not within the threshold range from the peakfocus value for the distance of the object and a determination that therespective focus value for the particular lens position does notcorrespond to the estimated local maximum of focus values.
 20. Themethod of claim 19, wherein determining the final lens positioncomprises: determining a start point for the fine search based on theplurality of lens position and the plurality of focus values;determining a direction for the fine search based on the plurality oflens position and the plurality of focus values; or determining a searchrange for the fine search based on the plurality of lens position andthe plurality of focus values.
 21. The method of claim 14, whereindetermining the distance of the object comprises performing anactive-depth assisted focus operation.
 22. The method of claim 14,wherein determining the distance of the object comprises performing acourse search.
 23. The method of claim 14, wherein determining thedistance of the object comprises performing phase detection auto focus(PDAF).
 24. The method of claim 14, further comprising: positioning,with the one or more processors and with a camera, the camera lens tothe final lens position; and capturing, with the one or more processorsand with the camera, an image with the camera lens positioned at thefinal lens position.
 25. The method of claim 24, further comprisingoutputting, with the one or more processors, the image at a display. 26.The method of claim 14, further comprising causing, with the one or moreprocessors, the camera lens to move to each of the plurality of lenspositions in response to determining the plurality of lens positions.27. A computer-readable storage medium having stored thereoninstructions that, when executed, configure one or more processors to:determine data associated with a distance between an object and a cameralens; determine a plurality of lens positions of the camera lens basedon the data associated with the distance between the object and thecamera lens; determine, for each one of the plurality of lens positions,a respective focus value to generate a plurality of focus values,wherein the instructions that cause the one or more processors todetermine, for each one of the plurality of lens positions, therespective focus value further cause the processor to determine, foreach one of the plurality of lens positions, phase differenceinformation; and determine a final lens position based on the pluralityof focus values.